Regression problems involve the prediction of a continuous, numeric value from a set of characteristics.
In this example, we'll build a model to predict house prices from characteristics like the number of rooms and the crime rate at the house location.
We'll be using the pandas package to read data.
Pandas is an open source library that can be used to read formatted data files into tabular structures that can be processed by python scripts.
In [1]:
# Make sure you have a working installation of pandas by executing this cell
import pandas as pd
In this exercise, we'll use the Boston Housing dataset to predict house prices from characteristics like the number of rooms and distance to employment centers.
In [2]:
boston_housing_data = pd.read_csv('../datasets/boston.csv')
Pandas allows reading our data from different file formats and sources. See this link for a list of supported operations.
In [3]:
boston_housing_data.head()
Out[3]:
In [4]:
boston_housing_data.info()
In [5]:
boston_housing_data.describe()
Out[5]:
After reading our data into a pandas DataFrame and getting a broader view of the dataset, we can build charts to visualize tha "shape" of the data.
We'll use python's Matplotlib library to create these charts.
Suppose you're given the following information about four datasets:
In [6]:
datasets = pd.read_csv('../datasets/anscombe.csv')
for i in range(1, 5):
dataset = datasets[datasets.Source == 1]
print('Dataset {} (X, Y) mean: {}'.format(i, (dataset.x.mean(), dataset.y.mean())))
print('\n')
for i in range(1, 5):
dataset = datasets[datasets.Source == 1]
print('Dataset {} (X, Y) std deviation: {}'.format(i, (dataset.x.std(), dataset.y.std())))
print('\n')
for i in range(1, 5):
dataset = datasets[datasets.Source == 1]
print('Dataset {} correlation between X and Y: {}'.format(i, dataset.x.corr(dataset.y)))
They all have roughly the same mean, standard deviations and correlation. How similar are they?
This dataset is known as the Anscombe's Quartet and it's used to illustrate how tricky it can be to trust only summary statistics to characterize a dataset.
In [7]:
import matplotlib.pyplot as plt
# This line makes the graphs appear as cell outputs rather than in a separate window or file.
%matplotlib inline
In [8]:
# Extract the house prices and average number of rooms to two separate variables
prices = boston_housing_data.medv
rooms = boston_housing_data.rm
# Create a scatterplot of these two properties using plt.scatter()
plt.scatter(rooms, prices)
# Specify labels for the X and Y axis
plt.xlabel('Number of rooms')
plt.ylabel('House price')
# Show graph
plt.show()
In [9]:
# Extract the house prices and average number of rooms to two separate variables
prices = boston_housing_data.medv
nox = boston_housing_data.nox
# Create a scatterplot of these two properties using plt.scatter()
plt.scatter(nox, prices)
# Specify labels for the X and Y axis
plt.xlabel('Nitric oxide concentration')
plt.ylabel('House price')
# Show graph
plt.show()
We could see in the previous graphs that some features have a roughy linear relationship to the house prices. We'll use Scikit-Learn's LinearRegression to model this data and predict house prices from other information.
The example below builds a LinearRegression model using the average number of rooms to predict house prices:
In [10]:
from sklearn.linear_model import LinearRegression
x = boston_housing_data.rm.values.reshape(-1, 1)
y = boston_housing_data.medv.values.reshape(-1, 1)
lr = LinearRegression().fit(x, y)
lr.predict(6)
Out[10]:
We'll now use all the features in the dataset to predict house prices.
Let's start by splitting our data into a training set and a validation set. The training set will be used to train our linear model; the validation set, on the other hand, will be used to assess how accurate our model is.
In [11]:
X = boston_housing_data.drop('medv', axis=1)
t = boston_housing_data.medv.values.reshape(-1, 1)
In [12]:
# Use sklean's train_test_plit() method to split our data into two sets.
# See http://scikit-learn.org/0.17/modules/generated/sklearn.cross_validation.train_test_split.html#sklearn.cross_validation.train_test_split
from sklearn.cross_validation import train_test_split
Xtr, Xts, ytr, yts = train_test_split(X, t)
In [13]:
# Use the training set to build a LinearRegression model
lr = LinearRegression().fit(Xtr, ytr)
In [14]:
# Use the validation set to assess the model's performance.
# See http://scikit-learn.org/stable/modules/generated/sklearn.metrics.mean_squared_error.html
from sklearn.metrics import mean_squared_error
mean_squared_error(yts, lr.predict(Xts))
Out[14]:
What kind of enhancements could be done to get better results?